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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the technique of detecting the structure of an 

eye from a portrait image. 

[0002] 

[Description of the Prior Art] From a portrait image, it explains based on reference "the 
description line extraction from the line drawing image of a face" (the Okada, Kiriyama, 
Nakamura and ****, and Kogakuin University research report No. 48 April, Showa 60) as the 
upper eyelid line of an eye, and an example of the conventional method which detects an eye's 
inner corner and an outer-canthus point. 

[0003] In drawing 2 , it supposes that the input image 31 with which near the eye is photoed is 
given, and a x axis and the y-axis are made into the axis of abscissa and the direction of an axis 
of ordinate of this drawing, respectively. At this time, the primary differential operator of a x axis 
with the property according to the property of an eye and the direction of the y-axis is applied 
to the above-mentioned input image 31. This primary differential operator is suitable for 
detecting the ** -> dark in an image, or change to dark -> **. The pixel from which this operator 
output serves as the maximal value by the above-mentioned x.y-car shaft orientations, 
respectively is used as the edge of each of these shaft orientations, and that output is given by 
the above-mentioned primary differential operator s output reinforcement. 

[0004] Label attachment of 8 connection is performed in the edge images of each of these x-y 
each shaft orientations, and a segment is constituted. Near 8 pixel a1 when setting an attention 
pixel to aO in drawing 3 with constituting a segment from 8 connection — It is the technique of 
judging whether an edge pixel exists in a8. Moreover, it asks for an average and distribution of 
edge reinforcement for this every segment. When an average and distribution of this edge 
reinforcement are small, it thinks that the segment does not constitute an upper eyelid, binary- 
ized processing is performed with a suitable threshold, and drawing 4 and drawing 5 which are 
the edge binary image of x-y each shaft orientations are obtained. 

[0005] Here, an upper eyelid line consists of one edge segment, and I think that this edge 
segment is convex and is carrying out characteristic distribution whose concentration 
distribution inside the closed curve made from the straight line which connects the endpoint and 
endpoint of this segment and this segment contains many for dark parts, such as a pupil and an 
eyelash part. Moreover, I think that an eye's inner corner and an outer-canthus point are both- 
ends points of the above-mentioned edge segment. Detection processing of an upper eyelid line, 
and an eye's inner corner and an outer-canthus point is explained to a detail below. 
(1) Extract a convex segment from drawing 5 . Here, the definition of things with a convex 
segment is defined as follows based on drawing 6 . 

[0006] (1a) If the longest distance found by the straight line passing through the endpoint and 
endpoint of a segment and the above (1a) which is perpendicular to it, finds distance with the 
point of the arbitration on a segment, and searches for the point used as the longest distance 
(1b) is located above the straight line passing through the endpoint and endpoint of a segment, it 
will be defined as convex. 
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(2) Set to 2a the die length of the segment which passes along the pixel [ 9th ] point from the 
endpoint and point of the direction in which an eye's inner corner is located in the segment of 
the convex on drawing 5 paying attention to one of them. Only a develops this segment further 
in the direction of an eye's inner corner, and an isosceles triangle as shown in drawing 7 of base 
2a and height 3a is set up. Furthermore, in this triangle, the segment which exists in the 
continuous-line section on drawing 7 which is below the horizontal position of an endpoint is 
extracted from the image of drawing 4 . 

(3) It is the straight line set up above (2) .11 It carries out and is 11. It is the straight line which 
passes along the point which separated 4 pixels from the intersection with the detected line right 
and left 12 It is 11 when it carries out. 12 The angle theta to make is searched for (refer to 
drawing 8 ). 

(4) If the line detected above (2) follows it, theta must be small to some extent. Then, threshold 
Ththeta of the include angle is defined suitably, and it is theta<=Ththeta. (1) 

The line to satisfy is extracted. When two or more segments detected above (2) exist, theta 
extracts the segment which is min. 

(5) Apply (4) from the above (2) to all convex segments. The processed convex segment and the 
segment extracted to each are connected, and turns into a candidate segment of an upper eyelid 
line. 

(6) Set up a closed curve as shown in drawing 9 by the straight line passing through an endpoint 
and an endpoint paying attention to one of the candidate lines of an upper eyelid line. 

(7) Frequency for high concentration value part more than threshold ThG given by the formula 
(2) within each closed curve (point section of drawing 9 ) It asks. 

[0007] 

ThG = (Gmax-Gmin)/5 (2) 

However, Gmax-Gmin is taken as the max in a closed curve, and the minimum concentration 
value. 

(8) Extract the line by which the frequency of the high concentration value calculated above (7) 
serves as max as an upper eyelid line. On the other hand, a setup of the candidate field of an 
eye is redone noting that an upper eyelid line does not exist, when all lines do not contain a part 
for high concentration value part at all. 

(9) Remove the wrinkle of the outer canthus detected superfluously from the detected upper 
eyelid line. It is perpendicular to the straight line passing through the endpoint and endpoint of an 
upper eyelid line, and it, distance with the point of the arbitration on a line is found, and the point 
used as the longest distance is searched for. 

(10) Find the same distance as the above (9) by the point searched for above (9) and the 
endpoint by the side of an outer canthus. From the longest distance point to the endpoint by the 
side of an outer canthus is removed (refer to drawing 10 ). 

The purpose was reached by making into an upper eyelid line the segment acquired by the above 
processing, and making the both-ends point into an eye's inner corner and an outer—canthus 
point. 
[0008] 

[Problem(s) to be Solved by the Invention] Many parameters which must be adjusted in order to 
operate well the conventional detection approach of of said upper eyelid, and an eye's inner 
corner and an outer-canthus point to the given image existed. Moreover, the processing itself is 
complicated and adjustment of the parameter at the time of not operating well is difficult. 
Moreover, the outer-canthus point was smoothly connected with the wrinkle only for the edge 
information used here, and the judgment of being an outer-canthus point was weak to the 
conditions of lighting etc. in order not to obtain ****** to delicate decision. Moreover, although 
the eye's inner corner and the outer-canthus point have been acquired on the conditions near 
the curvature of a segment, depending on the location of an eyelid with a fold, the detection 
result has been influenced greatly. 

[0009] The purpose of this invention is not influenced in the first place of lighting conditions etc., 
It is acquiring the value of the person proper of a processing object. The second purpose is 
permitting the effect by aging or the eyelid with a fold. 
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[0010] 

[Means for Solving the Problem] When the image which photoed near the eye of a person face is 
investigated in the direction of a jaw from the top of the head, the detection approach of the 
upper eyelid field of this' invention It is characterized by detecting the field where brightness is 
inserted the point of changing from ** tacitly, and in that it changes from ** to dark and **, and 
the width of face of this field being wider than the threshold on the basis of and detecting 

an upper eyelid field as a field where the average of the brightness value in this field is the 
darkest. 

[0011] Moreover, the detection approach of of the eyes inner corner, outer canthus, and upper 
eyelid field of this invention An upper eyelid field is detected from the image which photoed near 
the eye of a person face, the brightness on said image searches for the darkest point as 
compared with the surroundings, and the upper eyelid line candidate point group of an eye's inner 
corner and an outer-canthus point, and its neighborhood is detected. Said upper eyelid field, It is 
characterized by taking an OR with said upper eyelid candidate point group, and detecting the 
upper eyelid line of an eye's inner corner and an outer-canthus point, and its neighborhood. 
[0012] The detection approach of the structure of an eye of this invention detects an upper 
eyelid field from the image which photoed near the eye of a person face, and detects the 1st 
upper eyelid candidate point group of an eye's inner corner and an outer-canthus point, and its 
neighborhood from said image. Moreover, said upper eyelid field, Take an OR with said 1st upper 
eyelid candidate point group, and the 1st upper eyelid line of an eye's inner corner and an outer- 
canthus point, and its neighborhood is detected. Take the OR of the point that brightness 
changes from ** tacitly when said image is investigated from the top of the head, and said upper 
eyelid field, and the 2nd upper eyelid candidate point group is detected. The distance between 
the 1st upper eyelid line and said 2nd upper eyelid candidate point group is calculated toward the 
direction of the top of the head, and it is characterized by detecting the 2nd upper eyelid line as 
respectively nearest point group. 

[0013] Moreover, the point that the detection equipment of the upper eyelid field of this 
invention investigates the image which photoed near the eye of a person face in the direction of 
a jaw from the top of the head, and brightness changes from ** tacitly, A original field detection 
means to output the original field signal which is the field across which it faces in that it changes 
to ******** an d **, An upper eyelid candidate field detection means to output the upper eyelid 
candidate field signal which is a larger field than a threshold with the width of face of said Hara 
field signal, It is characterized by the average of the brightness value in said upper eyelid 
candidate field signal consisting of an upper eyelid field detection means to output the upper 
eyelid field signal which is the darkest field. 

[0014] Moreover, the detection equipment of the eye's inner corner, outer canthus, and upper 
eyelid field of this invention 1st eye structure candidate detection means by which consider as 
an input the upper eyelid field signal of an image and this image which photoed near the eye of a 
person face, investigate this image from a certain one direction, and brightness outputs the 1st 
eye structure candidate point which is the point of changing from ** with dark and **, 2nd eye 
structure candidate detection means by which investigate said image from the direction where 
said directions differ, and brightness outputs the 2nd eye structure candidate point which is the 
point of changing from ** with dark and **, The OR of said upper eyelid field signal, said 1st eye 
structure candidate point, and said 2nd eye structure candidate point is searched for, and it is 
characterized by consisting of an eye's inner corner and an outer-canthus point, and an OR 
means to output an upper eyelid line signal. 

[0015] Moreover, an upper eyelid detection means to detect an upper eyelid field from the image 
with which the detection equipment of the structure of the eye of this invention photoed near 
the eye of a person face, The 1st eye structure detection means which detects an eye's inner 
corner and an outer-canthus point, and the 1 st upper eyelid line from said image, 2nd eye 
structure candidate check appearance means by which investigate said image from the top of 
the head, and brightness outputs the 2nd eye structure candidate point as a point of changing 
from ** tacitly, An OR means to output the 3rd eye structure candidate point in quest of the OR 
of said upper eyelid field and said 2nd eye structure candidate point, The distance to the 
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direction of the top of the head between said 1st upper eyelid line and said 3rd eye structure 
candidate point is calculated, and it is characterized by consisting of an upper eyelid detection 
means to output the 2nd upper eyelid line as respectively nearest point group. 
[0016] 

[Function] The principle of this invention is detected in the description which shows an upper 
eyelid, an eye's inner corner, and an outer canthus below. 

- Make an upper eyelid:(1) upper eyelid into a boundary with other parts which have covered the 
eyeball and the eyeball fundamentally. 

(2) The upper eyelid is accompanied by the black eyelash. 

(3) Since the boundary of a black eyelash part and the part of a pupil is hardly in sight as for the 
upper eyelid of the part which is in contact with the pupil, make it into the boundary of a 
parenchyma top eyelash and the skin. 

(4) the upper eyelid of the part which is in contact with pewter — the boundary of an eyelash 
and the skin, or the boundary of an eyeball and other parts — both look good. Distinction of an 
endpoint is difficult continuously visible [ an eyelash / in many cases ] with the shadow of a 
wrinkle. For this reason, supposing the upper eyelid line an eye's inner corner and near an outer 
canthus is located in the part into which the eye's inner corner and the outer-canthus point 
extended far back very much (after-mentioned), it is seen in the direction perpendicular to ****, 
and is located in the darkest point. 

(5) Upper eyelids are wrinkles (duplex eyelid etc.) which run to two or more parallel. It often 
follows. 

[0017] In many cases, the eyelid with a fold has accomplished the thin line, and when it sees in 
the direction parallel to a line, it has not accomplished the edge. 

- An eye's inner corner:(1) eye's inner corner presupposes that it is located in the point that the 
boundary of an eyeball and other parts extended far back most. Therefore, suppose that it is 
located in the darkest point compared with the surroundings. 

- An outer-canthus:(1) outer canthus presupposes that it is located in the point that the 
boundary of an eyeball and other parts extended far back most. Therefore, suppose that it is 
located in the darkest point compared with the surroundings. 

[0018] The detection approach of the structure of the eye of this invention detects an upper 
eyelid, and an eye's inner corner and an outer-canthus point with the above-mentioned image 
description, and a parameter is little simple configuration and it realizes them as follows. 
[0019] The upper eyelid is accompanied by the eyelash, and near an eye, when a pupil is 
removed, it is the darkest field. Make the point where an eyelash starts at this time into the part 
where ** -> dark and brightness change from the top of the head, and let a field until it changes 
with the deepest ** -> dark -> ** of an eyelash be the candidate field of an upper eyelid. Point 
that ** -> dark and brightness change here when it sees in a certain direction Step(-) Edge, ** 
-> dark -> **, and point of changing V-Ditch Edge It will call. Moreover, field across which it 
faces with these two edges It will be called a Step (-) - V-Ditch field. The above-mentioned 
edge and a field are shown in drawing 1 1 . Step(-) Edge Secondary differential can detect [ the 
primary differential of an image ] as a point of 0 by negative. To this appearance V-Ditch Edge 
The primary differential of an image can detect as a point forward in secondary differential by 0. 
Step (-) - V-Ditch Edge The primary differential of an image can detect as a field forward in 
secondary differential by negative. The upper eyelid and pupil out of the above-mentioned upper 
eyelid candidate field can be easily distinguished by the width of face of a field. That is, in a field 
larger than the magnitude of a pupil, breadth makes the field where the average of the brightness 
value in the field is the darkest an upper eyelid field. The surroundings of an eye do not have 
change of big boom hoisting, and that of this upper eyelid field detection approach are strong to 
the effect of the shadow by change of lighting conditions. The above-mentioned upper eyelid 
field is equivalent to the upper eyelid field 32 to the input image 31 shown in drawing 2 . 
[0020] In case it asks for the upper eyelid line of an eye's inner corner and an outer-canthus 
point, and its neighborhood, as compared with the surroundings, the darkest point is searched for 
first, if it compares with the secondary differential which this corresponded with the location 
whose primary differential of an image is 0, and was used with the conventional technique — 
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more — low — since it is a degree, the effectiveness of being hard coming to win popularity the 
effect of a noise etc. will be acquired. Moreover, since the edge is not accomplished if an eyelid 
with a fold is seen from a direction parallel to a wrinkle, the data detected by this processing are 
not influenced not related at all. 
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t*aft*ci#iilttta*. H2«:SKLteA*Bi*3 1 
«:»UTIB«iS©±*-fe/t««. ^2cD±^^fc03 6 

[0 0 2 2 ] 

[0 0 2 3 ] BWjfi*i8BK3nri»4A*HIWt#l 1 
*na*6H*Sl«:BI^4 Step (-)-V-Di 

t c h mimm^m mwmm^so uc**) s 

tep (-)-V-Ditch fMlft 1 2 *S 
|g Step (-)-V-Di tch ««ft^ 1 2 

±*-S5yt««««S^ 1 3 t Itit^iWffl 
^S2tc<J:0t#^ o WEJh*Afc«li«Wfll*l 3©« 
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[0 02 4] Hate Step (-)-V-Di tch 

mm&m^m i o>-mmm&* a 1 4 *«i»TK?Br 

& 0 A^iH&fl-^ 1 UA^iT^, KH«**4— * 
[6jCCOl>T 1 1 ^C«»Bi»«»*S4 1 fC 

<to i 5 mzAjimmm^ i 

4, 2*»»Hi«ai**a4 2«:«fcD2^{R»iil*«# io 

5 2*»*. mis i ^cafc»BMR«^5 ii«nE2»a»» 

■HM»5 2**ft«l2«fc#R4 3. 4 4-C2llt 
U l?fc«#2fflHi*fi#5 3, 2#®#2ffiili«lfl^f 
5 4 £ff£o SulB 1 2 ttHMMfrV 5 3 ^IftSMK 

»*R4 5 l^at»2mW«BH»«^5 5* 
f#£ B B[l^«»2ttlWi*HI«fi#5 5£, 1MB 2^ 
a^2{iliij^{i-^5 4©t&Sffi^tt#*r4 Step 

(-)-v-Ditch w*ws«*«4 8fcj:9, s 

tep (-)-V-Ditch SR^HHIMI^S 6 £t# 

9, Step (-) —V- D itch 20 

[0 02 5] -x^mwomm • §k • ±*.&fc««w> 
i£tn^K£, si 2*fflo>T»iBT9rr*. 

[0 02 6] A«0Bttifi*^OfcA#B!fc«# 1 

i ±mm&<D±*&tcffims^ 1 4£a^£ u tsui^ 

^laJrfP^M^S^l© V-Ditch Ed 
ge^S (*l©B«Ji«lll«U#S) 4CC<fc»K 
flOV -Ditch Edgeft-^1 5*S*. £tt 
B5SBlB«*frsB*|Pj <k Stt £— & Jg 2 C7) V 
-Ditch Edge ttffi^S <»2<DB*jB«» 30 
&m^&) 5«:<fc?K 82© V-Ditch Edg 
e {f#lB*»*. 

[0 0 2 7 ] «re±*-&fc«««# 1 4 i «TBB» 1 <D 
V-Ditch Edge ft-S| 1 5 £fufBJfl2CD V 
-Ditch Edge fg# 1 6 CDIflSfl] @ 

si • ssi*±±£&tcmm i 7 &m 1 ©nift^ae 

3^6f#S, S6B • BJRi±*A/ctftffiiS*2 2£^rr 
[0 02 8] SuIB V-Ditch Edge ^tH^ 

S4, $<D—mmmz, ii5^tt»^c a^ 40 

BHSKf-sfl 1 £A^£-T£, KW££&£-#frtcot,> 
T 1 1 ^«»Hi«»ll*a4 1 K: «fc 0 1 

»#iIfflMa#5 1 £f#£ 0 lalBA^Hl^fi-^ 1 1 ^Kti 
t?Z>. m%ttft£Wifafcr>{<*XZ'X$kft'$'Z>, 2*X 
8fc»iB«iSiSt*S4 2 OCcfc 0 2 ^ca^IS»M# 5 2 £t# 
£ 0 tufB 1 #aWWBfc«# 5 1 * 2 fflKfc#S4 3 V 2 fit 
fbU l^«»2«ia<ftS#5 3*»S. «E1^C«»2 

«a®^5 3<D«if«cc*5wsfrfB2^a^ia««^5 2 

£t)^ iE-C*tl« V-Ditch Edge 
ff-sf 5 7£[fJ2rr£ V-Di tchEdge«f 50 
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©4 7 0Cj: i 3, V-Ditch Edge t£tH#© 

[0 02 9] CCm */JvBUffiitt*<t5«:. *ISI<D 
Stt* V-Ditch Ed g etfcHJ^S*:2o«# 
^*>tfTfli(r»TC>S#* ECC3rffl<DA&£ V-Di t 
chEdge fcttJ*R*W*in*T*>cfcl». 

[0030] ^ccc**?aa)a©«JS<D*HJ«B*. 0 1 

3*JBt>Tl»i'*-S. 

[0 0 3 1 ] AtJM<Dg«fi€:miJ L//cA^ili^«^ 1 
l*A*tr*±**fc«i(W>*a«B2 1#>6±£.& 

m<Dtfi&gim2 2CC<fc0fJIBiB«3^6BS • IR.St± 

[00 3 2] IWBIH«*BHtt&iei^6IH^* Step 
(-) Edge tftffi*«7K:J:«3 Step(-) 

Edge mms £f#£„ mz±&&tcffii&m^ 1 

4£buIB Step (-) Edge (t# 1 8 *m 

?#^o SuiBgsi • ajR*i±*-S5fcta«-^ 1 7 £tufB± 

tCOCvCHilU «feifiC*jS»iL't±*Afcai*#2 
0 £±£ -&fc*SliJ*S 9 CC <fc 0 f#6 B<Dfft®<D*fctti^g 

[0 0 3 3 ] I93B±* Afc^©1ftttJ§SB2 1 

ffiSg2 2W\ *IH8©BBB • @K * ±£&tcmm<D& 
[0 03 4] buIB Step (-) Edge ^tt!^© 
\ *KJl£?Zs *rtl«:^lr^ 1« 

a^-r^, 1 *a»iB*»»*a4 1 &c<fc 0 1 ^cfK^Bi 

l£f#£o SulBA^iiJ^ff # 1 l^A*i*T 

iW8*iaiiH*iaiK:o(,*r2^c«»"rs. 2%flt^ 

B»ifW*S4 2 (C J: 0 2 ^a^ffl^fi-^5 2 £f#£o 
mffl? 2 yc«»B««# 52^2 <B{b#K4 4T2«b 
2^«»2fflB««#5 4*»*. tt2 3C&&2fl[ 
5 4 CD«»3»iC*J W 1 ^a^iSIWt^ 5 1 CD 

ft^&IM^. iE-C*tltf , Step (-) Edge 
Edge ft^58^tH^f^ Step (-) Ed 
ge «8(#®4 8«C<fc9, S t ep (-) Edge 

[0 03 5] 

[0 1 ] *ISflJCD±^^/cfl^cDeiW^ScD-^Sfe^lcD 
[0 2] &^a£&6A^lIHitr&£ 0 





(6) 
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[0 3] 8aa6«cJ:S»»®«J5Effi©tt?HBI"C*4. 


* 


1 1 






[04] ft&^&t£i6VZ±£&tcMffittif8xM'JTfa^ 




1 2 


Step ( - ) —V - D i 


tch m^ciM^T 






1 3 






[0 5] ffi*^ffi«c^w4±*^yfc»auflaytt^i"ix 




1 4 










1 5 


Wl<D V-Ditch 


Edge ib-t 


[06] a^tfifcOrftSttWHt?**. 




1 6 


%$2<D V-Di tch 


Edge fg# 


[07] «3fe*ffi«cfew5«^Bfo«eHH-r**. 




1 7 


mm ■ msi£±£&tcm^ 




[08] se*^ffiec*$^4S^aii»<Dtt?8Hr*4o 




1 8 


Step(-) Edge 




[09] S£*#ffiCC*jWS±*i5^a#*60SiWHt? 




1 9 
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[010] S£*^ffiCC*5^S±S-fe/c*itH*ffi<DSiWBI 




2 1 










2 2 


mm ■ mm ■ ±.t^tc<D^mmm 


[011] Step (-) E d g e • V-D i tch 




2 3 
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3 1 
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3 2 
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3 3 
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3 5 






J&0r£>£ o 
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[014] Step (-)-V - Ditch Mi^ffi 
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6 0 1 ©i&SW^S 
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